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(57) A device (1 1 0) which can be portable and there- 
fore used in conjunction with several different data 
sources is arranged for wireless capture of encoded 
non-perceptible data directly from a light emitting de- 
vice, such as a television (101) or other video and/or 
audio output device. The data is part of the ordinary pro- 
gram displayed by the device (1 01 ) and yet indiscernible 
by a viewer or listener. The indiscernible data may be 
information needed to order those services or products, 
including prices, delivery intervals, shipping details, 
coupon offers, and so-on. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry (114 or 118) arranged to 
output at least portions of the data (or data derived from 
the captured data) to a remote agent or device 
(1 20, 121), such as via a telephone call (via 1 40 or 1 09 
or 112) or a data transfer to a point of sale device. 




Printed by Jouve, 75001 PARIS (FR) 



1 



EP 0 71 3 335 A2 



2 



Description 

Field of the Invention 

This invention relates to a device and a method for 

the wireless capture of encoded non-perceptible data 
from a light emitting device such as a television or other 
video and/or audio output device, and subsequent use 
of the captured data to initiate a transaction, such as 
placing a telephone call and transmitting a message to 
a remote agent or automated attendant, or delivering the 
data to a point of sale agent directly or indirectly via a 
telecommunications connection, or to operate another 
device. 

Background of the Invention 

In a copending application entitled "System and 
Method of Capturing Encoded Data Transmitted Over a 
Communications Network in a Video System" filed on 
behalf of Isenberg and Tuomenoksa on March 1 8, 1 994, 
Serial No. 08/210802 and assigned to the same assign- 
ee as the present application, it was recognized that 
non-perceptible information can be encoded in a televi- 
sion program, captured in a "set top box", and thereafter 
used to make telephone calls. The set top box is an elec- 
trical device that is "hard-wired" directly in the circuit be- 
tween the video sen/ices network and the television set 
(as a stand alone unit or alternatively as part of the tel- 
evision set itself) and thus, unfortunately, is not portable. 
AlsO; the set top box is dedicated to one particular tele- 
vision (or other video source) and thus cannot be shared 
among several sources. Finally, the user of the set top 
box may be uncertain as to the nature of the information 
captured therein. 

Summary of the Invention 

In accordance with our invention, a device which 
can be portable and therefore used in conjunction with 
several different data sources is arranged for wireless 
capture of encoded non-perceptible data directly from a 
light emitting device, such as a television or other video 
and/or audio output device. The data is part of the ordi- 
nary program displayed by the device and yet indiscern- 
ible by a viewer or listener As an example, a television 
monitor may be located in a kiosk or other public area, 
and display a program advertising selected services and 
products. The indiscernible data may be information 
needed to order those sen/ices or products, including 
prices, delivery intervals, shipping details, coupon of- 
fers, and so-on. 

Capture can be effected by decoding subliminally 
modulated video broadcast signals, in a manner sug- 
gested in U. S. Patent 4,807,031, issued on February 
21, 1989. to R. S. Broughton, et al., the teachings of 
which are incorporated herein by reference. Alternative- 
ly, a modulated carrier, which, for example, can be an 



amplitude shift keyed (ASK) or frequency shift keyed 
(FSK) signal having its primary frequency components 
atafrequency (e.g. 25 kHz) not usually present in a con- 
ventional television signal, can be added only to the lu- 
s minance component of a television signal, and the data 
can thereafter be captured using bandpass filtering and 
an ASK or FSK receiver. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry arranged to output at least 
10 portions of the data (or data derived from the captured 
data) to a remote agent or device, such as via a tele- 
phone call or a data transfer to a point of sale device. 

In accordance with the present invention, the en- 
coded non -perceptible data includes associated dialing, 
15 routing, or identification information, so that other por- 
tions of the captured data can be transmitted to an ap- 
propriate final destination or device. In this context, the 
final destination can be (a) a remote location, such as a 
platform arranged to process and consummate transac- 
20 tions, or (b) a point of sale system or other device (e.g. , 
TV, stereo, VCR Plus, smart mouse device, etc.). The 
other portions of the captured data that constitutes a 
payload to be communicated to the final destination or 
device, can, for example, be product or coupon informa- 
25 tion that is related to the display from which the non- 
perceptible data was captured, or programming or con- 
trol data. 

Further in accordance with the present invention, 
the capture device includes a built-in display, such as 
30 an LCD display, and the encoded non-perceptible data 
includes display information, so that at least portions of 
the captured data can be discerned by a user of the de- 
vice, and the user can thus be kept informed as to the 
operation of the device. The memory and the commu- 
35 nications circuitry in the capture device are advanta- 
geously arranged so that, in a transaction initiated by 
the device using captured information, a two-way dialog 
may be initiated between the user and an agent at a re- 
mote terminal or a POS system, and information may 
40 be transmitted to and stored in the device for later re- 
trieval. 

Brief Description of the Drawing 

45 The present invention will be more fully appreciated 
by consideration of the following Detailed Description, 
which should be read in light of the accompanying draw- 
ing in which: 

50 Fig. 1 is a schematic drawing of one embodiment 
of a system arranged in accordance with the 
present invention for wireless capture of encoded 
data transmitted with a television signal; 
Fig. 2 is a block diagram illustrating the components 
55 of capture device 110 of Fig. 1 ; 

Fig. 3 is a diagram illustrating the format of captured 
data; and 

Fig. 4 is a flow diagram of the process by which data 
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embedded in a subliminal visual or non-audible 
message is captured and used. 

Detailed Description 

Referring first to Fig. 1 , there is shown a schematic 
drawing of one embodiment of the present invention ar- 
ranged for the wireless capture of encoded non-percep- 
tible data from a light emitting device such as television 
101 , and subsequent use of the captured data to initiate 
a transaction which might be handled by a person at an 
agent terminal 1 20, or a person at another terminal 1 21 
that is supported by an interactive service platform 130. 

A television signal is received by television 101 via 
over-the-air transmission, as from transmission tower 
102 via a transmission link 103, or alternatively is sup- 
plied by a cable TV connection, or from a VCR/video- 
tape. The video portion of the television signal is dis- 
played on the picture tube or other visual display area 
105 of a monitor, computer or a television 101, while the 
audio portion of the television signal is played through 
or by projected from speaker 106. Associated with the 
program that is seen and/or heard by a viewer/listener 
is non-discernible encoded data that is transmitted as 
part of the audio and/or video signal. For example, the 
program may be an advertisement for a service or prod- 
uct, and the indiscernible data may create a subliminal 
visual pattern on the visual display area, that when prop- 
erly received and decoded, contains information need- 
ed to order those services or products, including prices, 
delivery intervals, shipping details, coupon offers, and 
so-on. 

Encoding, transmission and decoding of non-dis- 
cernible data may be accomplished as described in a 
patent application entitled "System and Method for En- 
coding Digital Information in a Television Signal" filed on 
behalf of T. Sizer and assigned to the same assignee as 
the present application, which co-pending application is 
incorporated herein by reference. Briefly, it was found 
using a spectrum analyzer on a typical video signal, that 
there are comparatively large frequency components at 
the line rate and at the frame rate and its harmonics, but 
that between these frequencies, there are other fre- 
quency bands in which little information is carried. One 
such frequency band is between 1 5 and 30 kHz. By add- 
ing a low level carrier signal or tone at a frequency in 
this band, say 25kHz, the video image is not degraded, 
but a properly tuned decoder can receive and decode 
the encoded information. In this way, digital information 
can be subliminally inserted in a video signal by adding 
to the video signal an amplitude shift keyed (ASK) or 
frequency shift keyed (FSK) carrier signal, and the dig- 
ital information can later be recovered using ASK or FSK 
decoding. Encoded data can also be inserted in a tele- 
vision signal and recovered by a receiver that responds 
to the picture displayed on the television, in the manner 
described in patent 4,807,031 cited above. 

The program displayed on television 101 is sensed 



by a portable capture device 110, which is described in 
more detail in connection with Fig. 2. When the non-dis- 
cernible encoded data is part of the video signaL such 
sensing takes the form of light from display area 105 
s being collected by an optical lens 117, while when the 
non-discernible encoded data is part of the audio signal, 
such sensing takes the form of sound from speaker 1 06 
being collected by a microphone (21 1 in Fig. 2). In either 
event, there is a wireless communication of the informa- 
10 tion from television 101 to capture device 110 such that 
the encoded data is "viewed" or "heard" rather than be- 
ing transmitted through a wired connection, or through 
a more conventional radio frequency communication. 
In addition to receiving non-discernible encoded da- 
15 ta, capture device 110 has the capability of (a) storing 
the encoded data, (b) transmitting (or otherwise output- 
ting) the encoded data (or information retrieved using 
such data) to a remote location or device, and (c) dis- 
playing the encoded data to a user The data may in- 
20 dude information for initiating a transaction or originat- 
ing a telephone call, as well as additional information (e. 
g.; price and ordering information) associated with the 
television program being displayed on television 101. 
The remote location may be agent terminals 120 or 121 , 
25 interactive service platform 130, or a point of sale sys- 
tem, and the communication between the capture de- 
vice 110 and the remote location may be one-way or 
two-way. 

In the case where encoded data is to be output from 
30 capture device 1 1 0 via a telephone call, the call may be 
initiated and transacted in one of several ways, as illus- 
trated in Fig. 1. First, the data may be transmitted over 
a wireless link to an associated telephone base station 
109 that is, in turn, connected to a telephone line 111 
35 that is part of a telecommunications network 1 1 5. Com- 
munication between capture device 110 and base sta- 
tion 109 may be accomplished in a variety of ways; as 
an example, the same signaling and transmission can 
be used as is presently used in communication between 
40 a cordless telephone handset and its associated base 
station. Second, capture device 110 may be directly 
connected to a telephone line 112 through an interface 
that includes a standard telephone jack 114. Third, cap- 
ture device 1 1 0 may include the functionality of a cellular 
45 telephone, so that the data may be communicated to a 
remote location via a wireless call placed to a mobile 
telephone switching office (MTSO) 140, which is in turn 
connected to telecommunications network 115. 

As explained more fully in connection with Fig. 2, 
50 capture device 110 may include a dual tone multi-fre- 
quency (DTMF) transceiver arranged to generate a se- 
ries of audible tones. Accordingly, a fourth way to initiate 
a telephone call would apply portions of the data to the 
transmitter (tone generator) portion of the DTMF trans- 
55 ceiver. Capture device 1 1 0 may then be placed in juxta- 
position with the microphone portion of a conventional 
telephone handset, such that the audible tones gener- 
ated in capture device 110 are acoustically coupled 
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through the handset to receivers in telecommunications 
network 115. 

Capture device 110 can also output encoded data 
directly, without making a telephone call. This is useful 
in conjunction with a point of sale system or the like, 
which includes a docking station or other interface to 
which the capture device may be connected, or an IR 
receiving device or a device responding to DTMF. 

If an outgoing telephone call is initiated by capture 
device 110, the call can illustratively be routed through 
telecommunications network 115 directly to an agent 
terminal 1 20, or to an interactive service platform 1 30 to 
which is connected another agent terminal 121. When 
a connection is completed to platform 1 30, the call may 
be routed by a switch matrix 131 to a voice response 
unit 132 that operates under the control of a processor 
134. Information included in the captured encoded data 
can be provided to the attendants, or processor 1 34 can 
perform a look-up and retrieval operation in database 
1 31 , using encoded data as a key, in order to effectuate 
a desired transaction. 

For certain transactions, information connected 
with a transaction may also be transmitted from service 
platform 130 to capture device 110, either for display to 
a user, storage for later use, or output to a point of sale 
system or device. This information may for example, be 
confirmation information, indicating that a transaction 
has been effected, or details regarding a just completed 
transaction, such as ticket information relating to a travel 
reservation. 

Referring now to Fig. 2, there is shown a block dia- 
gram illustrating in more detail the components in one 
embodiment of capture device 110 of Fig. 1 . If the en- 
coded non-perceptible data is inserted in the video por- 
tion of a television signal, the images displayed on visual 
display area 105 are captured by optical lens 117 and 
an associated photodetector 203, which is arranged to 
supply an electrical signal to filter/receiver 215 repre- 
senting the image. On the other hand, if the encoded 
non-perceptible data is inserted in the audio portion of 
a television signal, the output from speaker 106 is cap- 
tured by a microphone 21 1 and an associated amplifier 
213, which is arranged to supply an electrical signal to 
filter/receiver 215 representing the sound energy. In ei- 
ther event, the encoded non-perceptible data is decod- 
ed in filter/receiver 215 in a manner consistent with the 
manner in which the original data was encoded. Thus, 
filter/receiver 21 5 can be arranged to perform the same 
functionality as the elements illustrated in Fig. 4 of the 
above referenced Broughton et al. patent, or, alterna- 
tively, the operation of filter/receiver 215 can be as de- 
scribed in conjunction with Figs. 4 or 7 in copending ap- 
plication of T.Sizer. In each instance, it is noted that cap- 
ture device 110 receives and recovers encoded data by 
"watching" or "listening to" the output of television 101 
(or another similar display device), and that the capture 
device 110 is notconnected to television 101 or the other 
display device by a cable or wires. 



The output of filter/receiver 215, representing the 
data captured by capture device 110, is applied to a mi- 
croprocessor 220, which is arranged to perform various 
data processing and control functions in coordination 
s with programs stored in an associated memory 221 as 
well as inputs received from a user of capture device 
110. Generally speaking, capture device 110 may oper- 
ate in an active or a passive mode. In the active mode, 
capture device 110 responds only to user initiated cap- 
ture commands. In the passive mode, capture device 
110 captures all of the data sent to it while in that mode. 
In either mode, captured data may be routed by micro- 
processor to memory 221 and stored for use at a later 
time, as determined by the user. The captured data may 
also be displayed on a display 225, such as a liquid crys- 
tal display, so that a user will be aware of exactly what 
information was captured. 

The control programs contained in memory 221 can 
implement numerous functions, which, broadly speak- 
ing, can be categorized as (a) functions relating to data 
capture, such as SAVE and AUTOSAVE functions, (b) 
functions relating to output of already captured data, 
such as DIAL and SEND DATA functions, (c) functions 
relating to memory manipulation, such as the RECALL 
and DELETE functions. The above enumerated func- 
tions are initiated when respective buttons 241 - 246 are 
activated, and the activation signal output from a button 
is applied to microprocessor 220 via a button interface 
240. Each of the functions is discussed below. 

When the SAVE function is initiated by activation of 
button 241, capture device 110 is in the active mode, 
and the encoded data contained in the current audio or 
video output of television 101 is captured. This is ac- 
complished, for example, by activating optical lens 117 
and photo detector 203, or microphone 211 and ampli- 
fier 21 3, for a relatively brief period of time, typically on 
the order of several seconds, so that encoded data as- 
sociated with the current program being displayed on 
display area 1 05 or played through the loudspeaker 1 06 
is processed by filter/receiver 21 5 and stored in memory 
221. A visual or auditory cue may be contained in the 
video or audio program, to indicate to a user of a capture 
device that there is data to be captured. When the SAVE 
function is activated, microprocessor 220 may be ar- 
ranged to control display 225 so as to indicate the cur- 
rent status, e. g. , that the capture device is in the process 
of receiving and capturing data. When the data has been 
captured, microprocessor 220 may be arranged to con- 
trol display 225 so as to actually display some or all of 
the data, so that the user is now aware of information 
that had theretofor been subliminal and indiscernible. 
The display can also indicate that the device is "aimed" 
correctly. 

When the AUTOSAVE function is initiated by acti- 
vation of button 244, capture device 1 1 0 is in the passive 
mode. In this mode, capture device 110 captures a se- 
ries of encoded data that may be contained in each of 
a series of programs that can be viewed/heard on tele- 
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vision 101. This is accomplished, for example, by acti- 
vating optical lens 117 and photo detector 203, or mi- 
crophone 211 and amplifier 21 3, for a relatively long pe- 
riod of time, typically on the order of several minutes or 
hours. Encoded data captured from the various pro- 
grams being displayed on display area 105 or played 
through loudspeaker 106 over a long period of time are 
processed by filter/receiver 215 and stored in memory 
221 . As with the SAVE function, when the AUTOSAVE 
function is activated, microprocessor 220 may be ar- 
ranged to control display 225 so as to indicate the cur- 
rent status, e.g., that the capture device is in the process 
of receiving and capturing data. 

When a user of capture device 110 desires to use 
any of the data stored in memory 221, including data 
stored a result of the AUTOSAVE function, the contents 
of memory 221 may be retrieved and displayed on dis- 
play 225 by initiating the RECALL function upon activat- 
ing button 245. This permits a user to be aware of ex- 
actly what information was previously captured and 
stored. Through this function, the user will be able to 
selectively peruse or scroll through the stored data, so 
as to retain or use selected portions thereof. Activation 
of button 246 initiates the DELETE function, whereby 
portions of the data stored in memory 221 are selectively 
erased. In order to conserve and thereby "recycle" to 
storage capacity of memory 221, microprocessor 220 
may also be arranged to manage the contents in mem- 
ory 221, so that the oldest information stored therein 
could be cycled out and deleted as new information is 
captured and stored. 

Turning now to consideration of the use of already 
captured data, it is an important functionality of capture 
device 1 1 0 to permit such data to be output upon a com- 
mand from a user. Such data, when output, can be used 
to (a) make a telephone call, and, when the call is an- 
swered, to provide information to a live or automated 
attendant that receives the call, or (b) initiate a transac- 
tion, such as by interacting with a special purpose ter- 
minal or kiosk arranged to receive the stored data, or a 
device such as a VCR Plus or a stereo. In accordance 
with the arrangement shown in Fig. 2, data output via a 
telephone call can be initiated in several ways. 

First, captured data can be routed by microproces- 
sor 220 from memory 221 to a modem 246 and then to 
a base station interface 250, so that the information can 
be communicated to telephone base 1 09 of Fig. 1 . This 
in turn allows a conventional telephone call to be origi- 
nated from telephone base 109. 

Second, captured data can be routed through mo- 
dem 246 to a wire interface 253, which allows capture 
device 110 to be physically connected to a terminal or 
docking station for information off loading. 

Third, captured data can be routed to a DTMF trans- 
ceiver 247, which includes a tone generator arranged to 
output DTMF tones. These tones can be directly con- 
nected to a telephone line 1 1 2 via a standard connection 
such as RT1 1 jack 1 1 4. Alternatively, capture device 1 1 0 



can be placed in juxtaposition with the microphone in a 
telephone handset, and the audible tones can be ap- 
plied and used much like an audible frequency autodi- 
aler. 

s Fourth, captured data can be routed through mo- 
dem 246 to a cellular telephone 252, which can initiate 
a cellular call. When a cellular connection is completed, 
the captured data can be output in encoded form or as 
audible tones. 

10 For each of the arrangements just described, a tel- 
ephone call may be originated using the DIAL function. 
This function is initiated by activation of button 242. With 
this function, dialing or routing information, which is part 
of the data captured by capture device 110 and now 
15 stored in memory 221, is retrieved. This information is 
then further processed, so that a telephone call can be 
initiated. In some instances, a call is originated by ap- 
plication of stored data to DTMF transceiver 247, so that 
audible tones can be output. In other instances, a data 
20 call can be initiated by appropriately formatting stored 
data. In yet other instances, such as when a cellular call 
is dialed, stored dialing information may be applied to 
an outgoing number register. The exact manner in which 
information is output from capture device 110 can vary, 
25 in accordance with several embodiments of the present 
invention. 

While the DIAL function is used to output certain 
captured information needed to initiate a telephone call 
(or other data output function), the SEND DATA function, 
30 initiated by activation of button 243, is used to output 
other portions of the captured data stored in memory 
221, pertaining, for example, to a transaction that the 
user desires to make. This additional data is retrieved 
from memory 221 and appropriately transmitted, de- 
35 pending upon the manner and type of telephone call in- 
volved. Thus, data may be output either via base station 
interface 250 to telephone base 109, or via RJ 11 jack 
114 onto telephone line 112, or via cellular phone 252 
to MTSO 1 40. Note that if output occurs directly, without 
40 a telephone call, the SEND DATA function may cause 
information from memory 221 to be output via wire in- 
terface 253. During the SEND DATA function, the infor- 
mation being output may also be displayed on display 
225. This allows a person using the capture device to 
45 identify the information that was captured, and use the 
information in an actual transaction. 

It is to be noted here that capture device 110 of Fig. 
2 may optionally include an automatic speech recogni- 
tion circuit 245, an infrared (I R) transmitter 235, a hand- 
50 set 230 and a keypad 247. The purpose of I R transmitter 
235 is to allow captured data to be output directly to a 
device such as a point of sale system or a VCR or the 
like, via infrared transmission. The purpose of speech 
recognition circuit 245 is to convert captured data to spo- 
55 ken phrases, which can be played to a user or to an at- 
tendant after a telephone call is initiated. The purpose 
of handset 230 and keypad 247 is to allow capture de- 
vice 110, in addition to being used in accordance with 
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the present invention, to be used to complete conven- 
tional dialed telephone calls. 

In one application of the present invention, a tele- 
vised advertisement for a product includes additional or- 
dering or "coupon" information embedded in a sublimi- 
nal visual pattern within the video image displayed on 
visual display area 105. The coupon information de- 
scribes the item(s) offered for sale, the regular price, the 
coupon value, and contains additional information need- 
ed for ordering. This embedded information is transmit- 
ted between television 101 and capture device 1 1 0 at a 
rate such as 60 bits per second, that is sufficient such 
that the necessary information can be transmitted in a 
very brief period of time. In the application just de- 
scribed, the embedded product information, dialing 
number, coupon information and necessary checkbits 
together comprise approximately 2,000 bits of informa- 
tion, so that between three and four seconds would be 
required for transmission. Once the coupon information 
is stored in capture device 110, the user of the capture 
device can supplement the coupon information with 
user entered data which can be entered using keypad 
247. This supplemental information may include credit 
card or other billing information pertaining to the user, 
as an example. 

As shown in Fig. 3, the information represented by 
the encoded data can be organized in several fields. Us- 
ing the coupon application as an example, a first field 
301 contains a telephone number which may be used 
to originate a telephone call in order to order the product 
to which the coupon applies. Illustratively this will be a 
toll-free or "800" number. Other information associated 
with the coupon can include a product description in field 
303, an expiration date in field 305, the time that the 
advertisement was televised in field 307, the number of 
the television channel running the ad in field 309, and a 
serial number for the ad in field 31 1 , identifying either or 
both the particular ad and its sequence in an ad cam- 
paign. It is to be noted that various other information can 
also be included in the information illustrated in Fig. 3, 
such as keywords or attributes describing the received 
encoded data, as well as information indicating that a 
product is available in particular colors or sizes. The key- 
words would be useful in retrieving from memory 221, 
information relating to specific products or advertisers. 

The process by which information embedded in a 
subliminal visual or audible message is captured and 
used is illustrated in flow diagram form in Fig. 4. As in- 
dicated previously, two general modes of operation are 
contemplated, namely a "passive" mode and an "active" 
mode, both of which follow a "start" step 401 . 

In the passive mode, a user desires to operate cap- 
ture device 1 1 0 in an unattended manner, and to capture 
all of the encoded data that may be subliminally dis- 
played on a display device 105 or output from speaker 
1 06 over a relatively long period of time, typically several 
minutes or hours, until the user, at some later time, de- 
sires to discontinue or terminate the capture. This mode 



is initiated using AUTOSAVE functionality, by activating 
button 244. When this is done, a positive response is 
obtained in step 502, so that data capture, storage and 
display, all as described below in connection with step 

s 507, occur. While in the passive mode, capture is termi- 
nated by subsequent activation of the same button 224. 

In the active mode, a user operates capture device 
110 in real time, typically while viewing a video display 
or listening to an audio presentation. During the display 

10 or presentation, the user sees or hears a corresponding 
televised advertisement or other indication that imper- 
ceptible information is available for capture. The user 
may, at this time, be instructed to activate SAVE button 
241 , or the user may know, from previous experience, 

15 how to use the device. When button 241 is activated, a 
positive result occurs in step 503, thereby capturing the 
imperceptible encoded data, displaying all or a portion 
of the data on display 225, and storing same in memory 
221, all in step 507. Until the SAVE button is activated, 

20 capture device 110 may be in a "wait state" 505. 

The user of capture device 1 1 0 determines that the 
appropriate data has been captured by reviewing the in- 
formation seen in display 225. After the captured data 
has been reviewed and determined to be correct, acti- 
os vation of the DIAL button 242 on the portable capture 
device 110 in step 509 initiates step 51 3, by which stored 
dialing and/or routing information, illustratively in field 
401 of Fig. 4, is recalled from memory 221 . A call is then 
originated launched, such by transmitting the dialing in- 

30 formation from memory 221 through base station inter- 
face 250 to base station 109 and thence to telephone 
line 111 and telecommunications network 115 to inter- 
active sen/ice platform 130. The call may be a local or 
long distance call, made using a "plain old telephone 

55 service" (POTS) line; alternatively, a toll-free call may 
be launched using an 800 number. Desirably, automatic 
number identification (ANI), caller ID, and other features 
available in the telecommunications network may be 
used to provide additional information to the called party 

40 in addition to the captured data that is transmitted in suc- 
ceeding steps in the process. Until the DIAL button is 
activated, capture device 110 may be in a "wait state" 
511. 

After the call has been originated, a determination 
45 is made in step 515 that the call has been answered and 
thatthe SEND DATA button 243 has been pushed. (Until 
the SEND DATA button is activated, capture device 110 
may be in a "wait state" 517.) When a positive result 
occurs in step 515, information contained in fields 
50 403-411 is transmitted to the called destination in step 
519. As stated previously, from the point of view of the 
called party, the information received includes details 
concerning the advertised product, the identity of the 
caller, and other information associated with the item 
55 that was initially displayed or mentioned on the televi- 
sion program. Advantageously, the advertiser will know 
more about their customer than ever before. This is be- 
cause the advertiser will be able to determine from the 
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transmitted information which advertisement drew the 
business, how long it took the caller to respond to the 
ad. and the geographic location of the callers, based 
upon AN! or caller ID. It will be easy to tell which ads in 
a campaign are most successful. 

In accordance with one optional arrangement of the 
present invention, when data is captured and stored in 
step 507, the newly collected data is compared with pre- 
viously stored data. When a match is found, any dupli- 
cate data is dropped. In addition, the information in var- 
ious fields is examined, to assure that the data captured 
is in the appropriate format. If an error is detected in any 
data, the information may be deleted, and "new" data 
collected. This is accomplished by repeating capture 
step 507 several times. 

In another application of the present invention, an 
interactive television capability is provided in connection 
with a game show or television drama. For a game show, 
an audience would be asked to vote on various aspects 
of the game. This could include questions such as which 
person should go on a date with the contestant, or which 
singer is the best? The audience would be instructed to 
press the "SAVE" button on their capture device at the 
appropriate time specified for their choice. The viewers 
vote would then be cast by depressing the DIAL button, 
communicating with the game show host, and then 
transmitting information indicating the vote by depress- 
ing the SEND DATA button. Alternately one number 
could be transmitted and an automated interactive voice 
platform such as the InfoWorx™ platform available from 
AT&T could offer options to the caller. These calls could 
be toll free calls, or, alternatively, billed to the caller. 

In yet another application of the present invention, 
a video on demand capability can be provided. A viewer 
sees a video he or she would like to see advertised on 
a previewing network. An instruction to press the DIAL 
button on the capture device is made on the previewing 
channel to indicate to the viewer that he or she can press 
the button to order a particular video. The call would be 
placed to a voice response system used to reserve the 
video on demand. Information about which video the 
caller wanted to see is sent over the phone lines once 
the call is connected. The caller would merely have to 
acknowledge the order and select a time slot. 

The present invention can also be employed in the 
context of programming of a VCR, particularly where the 
user already has the capabilities provided by VCR Plus, 
a product that is commercially available. In this embod- 
iment, a code sequence associated with and represent- 
ing a particular television show constitutes the informa- 
tion included in a televised advertisement. When the ad 
is viewed, the viewer can press the SAVE button and 
send the sequences to the VCR Plus using IR transmit- 
ter 235. 

While the portability of capture device 110 is advan- 
tageous, it is to be noted that in some arrangements, 
the functions of capture device 110 and telephone base 
109 may be combined in a single stationary unit, which 



might resemble a cordless telephone base station. In 
this event, the functionality of capture device 110 would 
be included in the stationary unit, and that unit would be 
arranged, in the manner previously described, to collect 
s information from the visual display area 1 05 and to pro- 
vide storage of the collected information. In this arrange- 
ment, the stationary unit could also be adapted to utilize 
input devices such as optical character recognition 
(OCR) or scanners that would provide an additional 
method of inputting telephone numbers and coupon 
and/or product information. The hand piece of this sta- 
tionary unit would be used to place a call and send the 
data. 

The present invention is not limited to live televised 
programs, but can also be operated in conjunction with 
stored video programs played from a VCR. In this ar- 
rangement, advertisements and other encoded informa- 
tion is included in videos at the time they are recorded 
or at the time they are mastered and put onto video cas- 
settes. A viewer watching a video with such an ad can 
respond immediately to reserve another video, pur- 
chase one of the items advertised, or order catalogs au- 
tomatically. When a program containing subliminally en- 
coded data is viewed, the viewer simply presses the 
DIAL button, launching a telephone call, and then press- 
es the SEND DATA button, sending captured data, such 
as product information, to an agent, who can verify the 
order using the additional information that was captured 
or stored in the device. 

The portability of the present invention can be used 
to advantage in implementing what can, in effect, be 
called a "Roaming Bulletin Board". A user can be attend- 
ing a conference or traveling through the airport where 
kiosks or bulletin boards are set up. In the case of a con- 
ference, an attendee would walkthrough the conference 
viewing the displays. When the attendee sees a display 
of interest, he or she points the capture device towards 
the display, and captures the non-discernible encoded 
data from either the displayed image or the sound out- 
put. Later, the user launches a telephone call to the tel- 
ephone number specified and included in the captured 
data, and sends out coupon information that is also part 
of the captured data. 

It is further to be noted that the present invention 
can be used in the context of automobiles and mobile 
telephones. When a capture device in accordance with 
the invention is placed in a car, it can receive and cap- 
ture non-discernible encoded data contained in the au- 
dio output of the car radio. The captured data can in- 
clude a toll free or conventional telephone number for 
originating a mobile cellular telephone call from the car. 
The call may be placed to an interactive service plat- 
form, and additional captured data, such as coupon or 
similar information, can then be used to effectuate a 
transaction. 

Various modifications and adaptations of the 
present invention will be apparent to those skilled in the 
art. For example, the functions provided by the present 
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invention may be combined with tinose used in a per- 
sonal digital assistant (PDA) or a laptop computer. It de- 
sired, a simultaneous voice and data (SVD) modem of 
the type available now from AT&T could be provided in 
capture device 110 to allow communication both by 
voice and data transfer, at the same time. Various func- 
tions can also be added to the processes and applica- 
tions described. For example, telephone base 109 may 
be programmed to dial a number or series of numbers 
at specific times, when certain amounts of information 
or data have been collected, or based upon some crite- 
ria. Capture device 110 could then send collected infor- 
mation "in bulk" to interactive service platform 130. For 
this purpose, modem 246 could be used, so as to send 
the information more efficiently. 



shift keyed (ASK) or frequency shift keyed (FSK) 
carrier signal added to a television signal, and 
wherein capture is effected by detecting and decod- 
ing said carrier signal. 

5 

7. A system for capturing data comprising 

means for wireless capture of indiscernible 
data embedded in an audio/video display; and 
10 means for outputting at least portions of said 

captured indiscernible data to a remote device 

via a telephone call. 

8. Apparatus for initiating a transaction relating to a 
15 program appearing on a television display, said 

apparatus comprising 



Claims 

1. Apparatus for initiating a transaction, comprising 20 

means arranged for wireless capture of data 
directly from an audio/visual display said data 
being indiscernible by a viewer or listener, and 
means arranged to transfer at least portions of 25 
said data to a separate transaction processor 
to initiate said transaction. 

2. The invention defined in claim 1 wherein said data 
includes dialing information for originating a tele- 30 
phone call, and wherein said transfer means 
includes means for initiating said telephone call 
using said dialing information. 

3. The invention defined in claim 1 wherein said appa- 35 
ratus further includes 9. 

means responsive to said capture means for 
storing said data, and 

wherein said transfer means includes means 40 
for coupling information from said storing 
means directly to a separate transaction proc- 
essor. 

4. The invention defined in claim 1 wherein said cap- 45 
ture means is in a portable capture device and said 
transfer means is in a base station, and wherein 
said apparatus further includes means for transfer- 
ring information from said portable capture device 

to said base station. 50 

5. The invention defined in claim 1 wherein said 
audio/visual display includes non-perceptible 

encoded data, and wherein said capture means is 
arranged to receive and decode said encoded data. 55 10. 

6. The invention defined in claim 5 wherein said non- 
perceptible encoded data includes an amplitude 



means arranged for wireless capture of data 
directly from said television display, said data 
being indiscernible to a viewer of said television 

display; 

means responsive to said captured data for 
obtaining a telephone number, said telephone 
number being associated with a remote loca- 
tion arranged to perform transactions associ- 
ated with said program appearing on said tele- 
vision display; 

means for initiating a telephone call using said 
telephone number; and 
means for transmitting to said remote location, 
during said telephone call additional informa- 
tion contained in said captured data, said addi- 
tional information also pertaining to said pro- 
gram. 

A system for responding to a televised advertise- 
ment, comprising 

means for embedding non-discernible informa- 
tion in said advertisement, said information 
including (a) a telephone number for a remote 
processor arranged to process orders for the 
items displayed in said advertisement, and (b) 
other information relating to said items dis- 
played in said advertisement; 
means arranged for wireless capture of said 
non-discernible information directly from said 
televised advertisement; and 
means responsive to said captured information 
for (a) placing a telephone call using said tele- 
phone number and (b) transmitting said other 
information to said remote processor via said 
telephone call. 

A method for initiating a transaction, comprising the 
steps of 

capturing data directly from an audioA^isual dis- 
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play, said data being indiscernible by a viewer 

or listener, and 

transferring at least portions of said data to a 
separate transaction processor to initiate said 
transaction. s 

11. The method defined in claim 10 wherein said data 
includes dialing information for originating a tele- 
phone call, and wherein said transferring step 
includes initiating said telephone call using said io 
dialing information. 

12. A method for capturing data comprising 

capturing, using wireless capture, indiscernible 15 
data embedded in an audio/video display; and 
outputting at least portions of said captured 
indiscernible data to a remote device via a tel- 
ephone call. 

20 

13. A method for initiating a transaction relating to a pro- 
gram appearing on a television display said method 
comprising steps of 

using wireless capture data directly from said 25 
television display, said data being indiscernible 
to a viewer of said television display; 
responsive to said captured data, obtaining a 
telephone number, said telephone number 
being associated with a remote location 30 
arranged to perform transactions associated 
with said program appearing on said television 
display; 

initiating a telephone call using said telephone 
number; and 35 
transmitting to said remote location, during said 
telephone call, additional information con- 
tained in said captured data, said additional 
information also pertaining to said program. 

40 

14. A method for responding to a televised advertise- 
ment, comprising the steps of 

embedding non-discernible information in said 
advertisement, said information including (a) a 45 
telephone number for a remote processor 
arranged to process orders for the items dis- 
played in said advertisement, and (b) other 
information relating to said items displayed in 
said advertisement; 50 
capturing, using wireless capture, said non-dis- 
cernible information directly from said televised 
advertisement; and 

responsive to said captured information, (a) 
placing a telephone call using said telephone 55 
number and (b) transmitting said other informa- 
tion to said remote processor via said tele- 
phone call. 
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